Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@examples/auto_deploy/cookbooks/gemma_4_trtllm_cookbook.ipynb`:
- Line 160: The BASE_URL constant is set to a non-routable server bind address
("0.0.0.0"); change BASE_URL to use a routable client endpoint such as
"http://127.0.0.1:8000/v1" or "http://localhost:8000/v1" so client requests
target the local server correctly — locate the BASE_URL assignment in the
gemma_4_trtllm_cookbook notebook (the line containing BASE_URL =
"http://0.0.0.0:8000/v1") and replace the host portion accordingly.

In `@tensorrt_llm/_torch/auto_deploy/custom_ops/attention_interface.py`:
- Around line 903-905: When you grow the base cache tensor, also grow the
per-group cache tensors so their capacities stay in sync: in the branch that
calls self._input_buffer.resize("cache_loc", estimated_capacity) update every
cache_loc_g* buffer (the attributes created/used by register_window_groups,
e.g., cache_loc_g0, cache_loc_g1, etc.) to the same estimated_capacity (and do
the same mirrored update in the other resize block around lines 919-924). Locate
the places that call self._input_buffer.resize("cache_loc", ...) and add a loop
or explicit resizes to update each cache_loc_g* and any cache_loc_per_group
bookkeeping so staging (cache_loc_per_group) uses the new capacity.

In
`@tensorrt_llm/_torch/auto_deploy/custom_ops/attention/flashinfer_attention.py`:
- Around line 257-261: The decode-only path drops the sliding-window
(window_left) info, so wire it through: add a sliding_window/ window_left
parameter to prepare_flashinfer_metadata_host() and pass it into
plan_generate_only(), and then forward that value into
flashinfer.decode.fast_decode_plan() (or alternatively detect sliding_window and
route pure-decode batches to plan_decode()); update _to_flashinfer_window_left()
usage to compute the inclusive window_left and ensure
plan_generate_only()/fast_decode_plan() receive that value so SWA decode is
planned correctly.

In
`@tensorrt_llm/_torch/auto_deploy/custom_ops/attention/triton_paged_attention.py`:
- Around line 1151-1163: The current code overwrites seq_len_with_cache with a
window-local length, causing masking to mix global and local coordinates
(kv_page_offset, q_positions_2d, first_q_kv_pos vs kv_base_pos) and leading to
stale/skipped tokens after eviction; instead, preserve the absolute
seq_len_with_cache for masking and only derive a separate local length for
page-iteration bounds: compute cache_len_capped_local =
torch.minimum(cache_len_raw, max_cached) and seq_len_with_cache_local =
cache_len_capped_local + q_lens, keep seq_len_with_cache unchanged, and pass/use
seq_len_with_cache_local solely where page/local bounds are required by
_paged_context_kernel or page-iteration logic (leave masking code using
seq_len_with_cache).

In `@tensorrt_llm/_torch/auto_deploy/custom_ops/attention/trtllm_attention.py`:
- Line 337: The constant list in TrtllmAttention.get_constants() now includes
sink_token_length as the sixth positional constant, but the cached-op signatures
incorrectly place the output buffer parameter (out) between out_scale and
sink_token_length; update the cached-op call/signature(s) so that
sink_token_length remains in the constants prefix (i.e., keep sink_token_length
as the sixth positional constant returned by TrtllmAttention.get_constants())
and move the out parameter after the constants in the cached-op invocation;
apply the same fix to the other cached-op occurrences referenced around the
blocks at the other locations (the similar cached-op signatures near the later
ranges).

In `@tensorrt_llm/_torch/auto_deploy/shim/interface.py`:
- Around line 856-912: The code assumes managers is non-empty and will raise
IndexError for empty kv_groups; before constructing MultiPoolKVCacheManager or
indexing managers[primary_idx], check if managers is empty and handle the no-KV
fallback: if no managers, set self._kv_cache_manager to the appropriate fallback
(e.g., instantiate MambaHybridCacheManager or allocate local resources) instead
of creating MultiPoolKVCacheManager, ensuring the same fallback is used where
managers[primary_idx] would be accessed; update the branch that currently
assigns self._kv_cache_manager (the block that chooses between single manager
and MultiPoolKVCacheManager) to first handle len(managers) == 0, then
len(managers) == 1, then the multi-case, and reference the symbols
_kv_cache_manager, managers, primary_idx, MultiPoolKVCacheManager, and
MambaHybridCacheManager.

In `@tensorrt_llm/_torch/auto_deploy/utils/node_utils.py`:
- Around line 1052-1058: get_op_schema currently picks an arbitrary schema from
multi-overload packets which is non-deterministic; update get_op_schema to (1)
accept explicit types (hint op: Union[torch._ops.OpOverloadPacket,
torch._ops.OpOverload]) via type hints, (2) check for a single-overload
attribute `_schema` first and return it, (3) when `_schemas` is present prefer
and return the `"default"` key if it exists, and (4) if multiple schemas exist
and no `"default"` is present raise a RuntimeError describing the ambiguous
OpOverloadPacket instead of using next(iter(...)); keep the function name
get_op_schema and callers intact.

In `@tests/unittest/auto_deploy/singlegpu/shim/test_cached_sequence_interface.py`:
- Around line 1248-1261: The assertions depend on dict insertion order when
accessing interface._caches; instead, index the caches deterministically using
the resource names passed to add_resource ("kv_0" and "kv_1"). After
interface.initialize_resources(), replace the lookups that use
list(interface._caches.keys())[0/1] with direct access interface._caches["kv_0"]
and interface._caches["kv_1"] (used where kv_0 and kv_1 are assigned) so the
shape assertions reference the correct cache groups reliably.

---

Outside diff comments:
In
`@tensorrt_llm/_torch/auto_deploy/custom_ops/attention/torch_backend_attention.py`:
- Around line 432-456: The decorator for torch_backend_mha_with_cache
incorrectly declares mutates_args=() while the function mutates k_cache and
v_cache via _write_generate_kv_cache and _update_kv_cache; update the
`@torch.library.custom_op` on torch_backend_mha_with_cache to list the mutated
tensor arguments (k_cache and v_cache) in mutates_args so the op contract
matches the actual in-place updates and prevents torch.compile misoptimizations.

In `@tensorrt_llm/_torch/auto_deploy/shim/interface.py`:
- Around line 977-992: The returned KV stats and total_managed use only the last
group's kv_managed; update the aggregation to use the union across all groups
(kv_managed_all) instead. Replace usages of kv_managed when computing
total_managed and the "kv_managed" return value with kv_managed_all (i.e.,
compute total_managed = len(kv_managed_all) + ssm_managed_count +
conv_managed_count and return "kv_managed": len(kv_managed_all)); keep other
derived counts (paged_total, kv_total, paged_other, other_total) unchanged. Use
the existing symbols kv_managed_all, ssm_managed_count, conv_managed_count,
total_managed, and the return dict in this function to locate where to apply the
change.

---

Nitpick comments:
In `@tensorrt_llm/_torch/auto_deploy/shim/ad_executor.py`:
- Around line 857-862: The SWA eviction message currently uses ad_logger.info
and can be noisy; change the log level to debug in the SWA eviction block inside
ad_executor.py (the conditional that checks front_removed > 0 and i == 0) so
routine eviction details are emitted with ad_logger.debug instead of
ad_logger.info, keeping the same message text and context (group_idx,
window_size, request.py_request_id, len(all_indices), front_removed, num_active,
page_offset_g) to preserve diagnostic data.

In `@tensorrt_llm/_torch/auto_deploy/shim/interface.py`:
- Around line 47-165: MultiPoolKVCacheManager is missing type hints and public
docstrings; update the class and its public surface to be a proper drop-in
KVCacheManager replacement by adding PEP484 type annotations and short
docstrings: annotate __init__(self, managers: List[KVCacheManager], primary_idx:
int = 0) -> None and document parameters, add return types for properties (e.g.,
impl -> Any or the actual Impl type, tokens_per_block -> int, max_blocks_per_seq
-> int, blocks_in_primary_pool -> int), methods (get_num_free_blocks() -> int,
get_max_resource_count() -> int, get_needed_resource_to_completion(request:
RequestType) -> int or appropriate type, get_num_kv_blocks(num_tokens: int) ->
int, prepare_resources(scheduled_batch: ScheduledBatchType) -> None,
free_resources(request: RequestType, pin_on_release: bool = False) -> None,
update_resources(...)-> None, add_dummy_requests(request_ids: Sequence[str],
**kwargs) -> Any, shutdown() -> None, get_pool(group_idx: int) ->
KVCacheManager, num_pools -> int, max_concurrent_sequences -> int,
get_buffers(idx: int, kv_layout: str = "NHD") -> BufferType (or raise
NotImplementedError with a docstring explaining alternative),
event_buffer_max_size -> int, enable_block_reuse -> bool, enable_partial_reuse
-> bool, is_draft -> bool, kv_cache_pool_pointers -> PointerType,
kv_cache_pool_mapping -> MappingType, get_cache_indices(request: RequestType,
**kwargs) -> IndexType, store_blocks_for_reuse(request: RequestType, pin_blocks:
bool = False) -> None; include short docstrings on the class and each public
method/prop (at least __init__, get_buffers, get_pool, and get_cache_indices)
describing behavior and any divergence from single-pool KVCacheManager so
callers and type checkers can rely on it.

In `@tests/integration/defs/accuracy/test_llm_api_autodeploy.py`:
- Around line 982-984: EXTRA_EVALUATOR_KWARGS is defined as a mutable class
attribute; change it to a ClassVar-annotated constant or convert it into a
property to satisfy static analysis: annotate the symbol EXTRA_EVALUATOR_KWARGS
as ClassVar[dict[str, Any]] (import ClassVar and Any from typing) or replace it
with a `@property` that returns a fresh dict (e.g., def
EXTRA_EVALUATOR_KWARGS(self) -> dict[str, Any]: return {"apply_chat_template":
True}); update the declaration and imports accordingly.

In
`@tests/unittest/_torch/auto_deploy/unit/singlegpu/transformations/library/test_shared_kv_attention.py`:
- Around line 446-449: The test function
test_flashinfer_shared_kv_cached_attention_reads_aliased_cache_without_writing
uses an early return when CUDA is unavailable; replace that return with a call
to pytest.skip("CUDA is not available") so the test is recorded as skipped, and
ensure pytest is imported at the top of the test module if not already present.
- Around line 96-126: The variable `batch` in the `_manual_attention` function
is unused and triggers a static analysis warning; change the unpacking from
`batch, seq_len_q, num_heads, _ = q.shape` to prefix the unused variable (e.g.,
`_batch, seq_len_q, num_heads, _ = q.shape`) or otherwise rename it to `_batch`
to silence RUF059 and indicate it is intentionally unused; update any references
if you choose a different name.

In
`@tests/unittest/auto_deploy/singlegpu/custom_ops/attention/test_flashinfer_attention_op.py`:
- Around line 135-137: This test file repeats a long positional argument list
when invoking the FlashInfer attention op, which is brittle; create a small
local helper (e.g., call_flashinfer_attention or flashinfer_attention_helper) in
tests/unittest/auto_deploy/singlegpu/custom_ops/attention/test_flashinfer_attention_op.py
that wraps the actual op invocation and accepts either a partial kwargs dict or
the same parameters with sensible defaults, then replace each repeated call-site
(the clusters around lines shown in the comment) to call that helper instead;
update occurrences referenced in the review (around the groups at 135-137,
265-267, 396-398, 491-493, 625-627, 784-786, 892-894, 987-989) so future
arity/order changes only need updating in the single helper.

In
`@tests/unittest/auto_deploy/singlegpu/custom_ops/attention/test_triton_paged_attention.py`:
- Around line 493-496: The first assignment to q_pos (q_pos = torch.arange(s_k -
s_q + s_q, device=q.device)) is dead code because it is immediately overwritten
by the later assignment; remove that redundant line and retain the intended
prefill assignment (q_pos = torch.arange(s_k - s_q, s_k, device=q.device)) and
k_pos assignment (k_pos = torch.arange(s_k, device=q.device)); also update or
keep the inline comment to reflect that q_pos now represents absolute positions
for prefill.